Eliciting group judgements about replicability: a technical implementation of the IDEA Protocol

In recent years there has been increased interest in replicating prior research. One of the biggest challenges to assessing replicability is cost resources and time that it takes repeat studies. Thus an impetus develop rapid elicitation protocols can, a practical manner, estimate likelihood research findings will successfully replicate. We employ novel implementation IDEA ('Investigate', 'Discuss', 'Estimate' 'Aggregate) protocol, realised through repliCATS platform. The platform designed scalably elicit expert opinion about social behavioural science protocol provides structured methodology for eliciting judgements reasoning from groups. This paper describes as multi-user cloud-based software featuring (1) technical on replicability, (2) capture consent demographic data, (3) on-line training replication concepts, (4) exporting completed judgements. has, date, evaluated 3432 claims 637 participants.